Information Theory Tutorial Review of ‘To code or not to code - Lossy Source Channel Communication revisited’

In this tutorial, the paper ”To code or not to code Lossy Source Channel Communication revisited’ [1] is reviewed. This paper investigates the conditions under which a source-channel communication system is optimal. To test the optimality of a communication system, it is sufficient to measure the average cost and the average distortion, and check if these quantities lie on the optimal costdistortion tradeoff curve. The separation principle introduced by Shannon in his classic paper [2] states that is optimal to split the source compression and channel coding into two successive stages. Optimality is generally achieved by means (asymptotically) long codewords. However, the use of long codewords is not necessary. There exist cases for which uncoded transmission is optimal. For example, if a uniform binary source is plugged into a binary symmetric channel, an optimal communication system results(if the distortion is measured in terms of Hamming distance). The same holds if a gaussian source is connected to the input of a gaussian channel.The resulting communication system is optimal in the sense of mean square distortion. In these cases, we observe that optimality results from the ‘probabilistic matching’ of the source and channel statistics. The authors propose that this matching is the fundamental reason that the system achieves optimality and extend this idea to present a basis for communication systems that are optimal. The authors also look at cases of nonergodic and multiuser channels for which the source-channel coding approach gives better performance than the separation principle approach.